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Language and other background factors affecting secondary pupils’ 
performance in Mathematics in South Africa 

Introduction 

This research is a secondary analysis of the performance of the South African pupils in the 
Third International Mathematics and Science Study-Repeat (TIMSS-R) in which pupils wrote 
tests in mathematics and science, but in South Africa pupils also had to write an English test, 
which was included as a national option. TIMSS-R was conducted in 1998/1999 in South 
Africa and internationally under the auspices of the International Association for the 
Evaluation of Educational Achievement (lEA). South Africa was just one of the 38 countries 
participating in the study. However, the South African pupils performance was significantly 
below that of all other participating countries including other developing countries such as 
Morocco, Tunisia, Chile, Indonesia and the Philippines. This paper presents the final results 
of a three-year doctoral research project (Howie, 2002) and includes the final Partial Least 
Square analysis and multilevel analysis of the effect of language and other contextual factors 
on student and school level. 

South Africa faces the challenge of providing quality mathematics education for its multi- 
cultural society of 43 million people. The rich diversity of this society is reflected by the large 
number of official languages of the country. English is spoken as a first language by less than 
ten percent of the population and is the language of business and government. It is also one 
of two languages usually used at schools although it is not the most widely spoken language 
at home. However, the issue around the language policy for teaching and learning has 
become a sensitive and controversial topic in South Africa just as it is in many other post- 
colonial countries. 

Objectives of the study 

The research reported here concentrates on the final outcomes of the exploration of the 
performance of the South African pupils in mathematics and most especially the relationship 
between mathematics achievement and pupils’ proficiency in English. The factors relating to 
the pupils performance in mathematics and English language proficiency were explored in 
relation to the background information that was also collected from the pupils, teachers and 
principals of the schools included in the study. 

The following research questions addressed in this paper are: 

1. What is the effect of language on South African pupils performance in mathematics? 

2. How does the effect of language relate to other background variables collected at school 
and student level? 

The extent of the pupils’ aptitude in English and mathematics was analysed. Every South 
African pupil in TIMSS-R completed a standardised written skills and language usage test 
(HSRC, 1990). The performance of the pupils in this test indicates a measure of language 
proficiency in English. As English language proficiency in South Africa is correlated with 
other important background variables due to the nature of the country’s political past other 
background variables were explored and included in a multilevel model developed to address 
the second research question. 
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Literature Review 

A number of reports and articles have been written on the status of mathematics (and science) 
education in South Africa (Amott & Kubeka, 1997; Kahn, 1993; Taylor & Vinjevold, 1999; 
amongst others). Many have commented the poor results achieved in the mathematics 
matriculation examinations. However, there have been no national surveys on secondary 
pupils' achievement in mathematics conducted by the Department of Education or other 
government departments. Only in 2001, was the decision made to introduce sample-based 
national assessments at grades 3, 6 and 9 levels to provide policymakers with information 
regarding the effectiveness of the education system. Research conducted, that included 
achievement data, were case studies on local and regional samples (see Maja, Swanepoel & 
Du Toit, 1999; Monyana, 1996; Rakgokong, 1994; amongst others), which were not 
nationally representative samples resulting in limited inferences being made from the data. 

However, a number of factors have been reported pertaining to the poor performance of pupils 
in the matriculation examinations and in general (Adler, 1998; Amott & Kubeka, 1997; Kahn, 
1993; Monyana, 1996; Setati & Adler, 2000; Setati, Adler, Reid & Bapoo, 2001; Taylor & 
Vinjevold, 1999). These include: inadequate subject knowledge of teachers, inadequate 
communication ability of pupils and teachers in the language of instruction, lack of 
instructional materials, difficulties experienced by teachers to manage activities in 
classrooms, the lack of professional leadership, pressure to complete examination driven 
syllabi, heavy teaching loads, overcrowded classrooms, poor communication between policy- 
makers and practitioners, as well as lack of support due to a shortage of professional staff in 
the ministries of education. Most of these were reported on the basis of classroom 
observations and discussions with teachers and other stakeholders. Only Monyana (1996) 
collected data for the purpose of analysing the factors that had an effect on mathematics 
performance and utilised inferential statistics. However, there is a study involving data from 
100 schools as part of the Quality Learning Project that has also collected background data in 
order to infer reasons for pupils' performance in mathematics (and science) (see Prinsloo, 
Kanjee, Pfeiffer & Howie, 2001). In the Third International Mathematics and Science Study 
(TEMSS) conducted in 1995, the performance of the South African pupils was significantly 
below that of all the other 40 participants in the study (Howie, 1997; Howie & Hughes, 1998). 
TEMSS provided South Africa with the first national representative overview of how South 
African pupils were performing in mathematics (and science). However, there were 
significant language and communication problems with South African pupils learning 
mathematics in a second language. Pupils in all three Grades (7,8,12) showed a lack of 
understanding of both mathematics questions, and an inability to communicate their answers 
in instances where they did understand the questions. Pupils performed particularly badly in 
questions requiring a written answer (Howie, 1997; Howie & Hughes, 1998). 

Internationally, research addressing factors related to achievement in mathematics were found 
using data from, for example Belgium (van den Brock & van Damme, 2001), and Eastern 
Europe (Vari, 1997), but most were found in the USA (Sojourner and Kushner, 1997; Teddlie 
& Reynolds, 2000, amongst others). No studies were found either nationally or internationally 
that attempt to link English Language proficiency to mathematics achievement at secondary 
level using such a comprehensive dataset with data on pupil, class and school levels. 

Studies regarding the effects of language on mathematics achievement appear to indicate the 
importance of language in achievement generally, including mathematics (Berry, 1995; 
Clarkson, 1991;Tartre & Fennema, 1995; Young, 1997). There seems to be sufficient 
evidence internationally and some evidence locally to warrant the assessment of language and 
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its relationship to mathematics on a large scale in South Africa. 

Clearly there are many factors that have been found on school, class and student levels to 
have positive and negative effects on mathematics achievement (see Howie, 2002, for a 
comprehensive discussion). Here summary conclusions are presented whereby only 
references to African research and/or pertaining to less developed countries are included. 

The factors on student level discussed in the literature included: socio-economic status 
(Afrassa, 1998; Cherian, 1992; Howie & Pietersen, 2001, amongst others), books in the home, 
parental education, parents occupation (Afrassa, 1998; Eshetu, 1988, Gennet, 1991), parental 
relationships (Eshetu, 1988), parental press, parent's self-concept, pupils' attitudes to 
mathematics, family size (Behutiye & Wagner, 1995), jobs in the home (Deresse, Wagner & 
Alemaychu, 1990; Daniel, 1995)), pupils' aspirations, peer group attitudes, pupils' self 
concept (Howie & Wedepohl, 1997), self expectations, pupils' anxiety (Maqsud & Klalique, 
1991), enjoyment of mathematics, attitudes towards maths (Sayers, 1994), , reading ability, 
gender (Afrassa, 1998), age, attitudes towards teachers (Georgewill, 1990), time spent on 
homework (Afrassa, 1998). Of these most were investigated in this study as well. The 
exceptions are parents' occupations, parental relationships, parents' self-concept, pupil 
anxiety, cognitive ability, reading ability and attitudes to teachers. This is not because they are 
not important, but rather due to the limitations of the dataset that was used. 

On classroom level, factors found in the literature were the learning environment, teacher's 
characteristics (including gender) (Monyana, 1996), teacher's personality, streaming, 
computers, teachers' competence (Georgewill, 1990; Taylor & Vinjevold, 1999), teacher's 
confidence, education background, teacher's qualifications (Amott & Kubeka, 1997; 
Mpofana, 1989), teachers' methods, class size (Cohn & Ressmiller, 1987), time on task, 
disruptions in class, calculators, content coverage, and assessment. Of these factors, most 
were explored in the study, the only exceptions being teacher's personality and content 
coverage. The former not having been included in the questionnaires and the latter due to data 
problems where the data could not be recovered. 

Finally on school-level a number of factors have been investigated in previous studies. These 
included textbooks, teacher quality, time on task, leadership, organisation, management 
(Riddell, 1997), decision-making, within-school hierarchy, communication, school size, 
professional development (Cohn & Ressmiller, 1987), location, commitment, and the 
controlled environment. In this study, only textbooks, time on task, leadership, decision- 
making, school size and location were explored, as the other factors were not included in the 
data collection. 

Theoretical framework 

In order to address the objectives of the study, a conceptual model was developed for this 
research. This was to allow for the exploration of contextual factors within different levels 
influencing pupils’ achievement in mathematics within the context of South Africa. A 
summary of the model description is given here whilst a complete description can be found in 
Howie (2002). 

The model was informed largely by Shavelson, McDonnell and Oakes, 1987, as well as other 
literature discussed in the previous section. The model presents the education system in terms 
of inputs (including contexts), processes and outputs. The inputs are the policy-related 
contexts on a national, provincial and local level from which the intended curriculum (in the 
meaning of what should be taught in schools and learned by the pupils) is also designed and 
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developed. They also include the antecedents: the economic, physical and human resources 
supplied to different levels of the system; the characteristics of the teachers and the 
background of the pupils. Inputs into the system affect all the processes of education, which 
may also be seen as the practice in education. Different processes (relating to what is taught 
and how it is taught) take place within the districts, schools, and inside the classrooms in 
terms of the implemented curriculum (in the meaning of what is actually being taught in the 
classrooms), teaching (in the meaning of the context and conditions under which teachers 
work) and instruction. The outputs, also seen as the outcomes, eventuate in terms of the 
achievement of learners in specific subjects such as mathematics; participation in class and 
school activities, and finally learners’ attitudes towards subjects and schooling and aspirations 
for the future. It is expected that, due to the dynamics of the processes included in the model, 
there will also be indirect benefits and outcomes, such as improved learner participation partly 
due to improved curriculum quality. 

The model serves as an important theoretical and conceptual basis for the analysis of the 
TMSS-R data. As the data were collected on a number of education levels, namely, school, 
classroom and learner level, the model serves as a guide to explore the causal links for the 
learners’ achievement. 

Research design 

The research was divided into two phases. Phase one concentrated on describing the South 
African pupils’ performance in mathematics and provided descriptive information regarding 
the background characteristics of the pupils, their mathematics teachers and the schools that 
they attended. However, this is not the focus of this paper and can be found in Howie (2002). 

The second phase was an exploratory and analytical phase of the study focusing on the 
secondary analysis of the TIMSS-R data related to mathematics achievement and a summary 
of the methods as well as the findings are discussed in this paper. The data were explored to 
investigate the reasons for the pupils’ performance and to explore the inter-relationships of 
achievement and the background variables revealed by pupils, teachers and the school 
principal. In particular, the exploratory part of the study was to determine the factors that 
influence mathematics achievement and performance of South African pupils and to ascertain 
the effect of South African pupils’ language and communication skills on their achievement 
in mathematics. 

Sample 

The TIMSS requirements stipulated that a minimum of 150 schools be tested and that a 
minimum of one class (preferably one whole class) per school be tested. The South African 
initial sample was expanded to 225 to accommodate the inter-provincial analysis required. 
The nationally representative sample drawn stratified by province, school sector and medium 
of instruction. A two-stage stratified cluster sample of 225 schools was randomly selected 
and stratified according to province, type of education (government or private) and medium of 
instruction (English and Afrikaans). Tests and questionnaires were administered (by the 
Human Sciences Research Council based in Pretoria, South Africa) to more than 9000 pupils. 
Questionnaires were also administered to 200 school principals and 400 teachers of 
mathematics and science at Grade 8 level. After the data cleaning a representative sample of 
194 schools and 8146 pupils was used in the data analysis. 



ERIC 



6 



5 



Instruments 

The following instruments were designed and implemented for TIMSS-R; Eight test booklets 
containing mathematics and science achievement tests, pupil questionnaire, mathematics 
teacher questionnaire, science teacher questionnaire, school questionnaire. The mathematics 
test can be considered valid for South Africa, as an analysis by South African national 
department officials showed that 83% of the achievement test items were covered in the 
national curriculum for grade 8, whilst it appeared that in all countries (including South 
Africa) pupils performed similarly on the test items covered by their national curriculum as 
as on the part of the test not covered in the curriculum (Mullis et al., 2000). 

As a National Option for South Africa an English language proficiency test was included 
specifically for South African pupils. This instrument had previously been validated by the 
Human Sciences Research Council and standardised for Grade 8 Second Language pupils in 
South African schools (HSRC, 1990). At the time of the TIMSS-R study, this test was the 
only standardised South African second language test at the Grade 8 level that could be found. 
Two language education experts evaluated the test and felt that it was still valid. Questions 
were also included in the TEMSS-R pupils’ and teachers’ questionnaires, to ascertain the 
extent and level to which the pupils are exposed to English. They included pupils’ home 
language, ethnic group, the language spoken predominantly by the pupils in the mathematics 
class, the language used by the mathematics teacher in class, media languages pupils are 
exposed to and the language of their reading materials. In this research, data from the test 
booklets, pupil questionnaires, mathematics teacher questionnaires, school questionnaire and 
the national option were analysed. 

Data Analysis 

The first step in the analysis plan was to produce frequencies of all the possible school-, class- 
and student-level factors linked to the research questions and the data was explored to make 
constructs (such as possessions in the home) and thereafter a correlation matrix was made. In 
the descriptive phase bivariate and multivariate analyses were also done. The correlation 
matrix was important to identify possible variables linked to achievement, to build constructs 
and to prepare a basic model for further analysis. 

Given that there is a number of variables reported to influence pupils achievement as well as 
the vast number of variables in the database, and that some of these were intricately inter- 
related, Partial Least Square analysis (PLS) (Sellin, 1989) was used initially to analyse those 
student-level and classroom-level factors that influenced pupils’ achievement in mathematics. 
Partial Least Square analysis is a form of multivariate analysis that explores causal relations 
amongst different variables (factors) and represents these by means of graphic path diagrams 
(see e.g. Figure 3). These diagrams or causal models illustrate the ‘paths’ along which the 
causal influences flow. This type of analysis allows researchers to estimate or predict both the 
direct and indirect effects of a set of independent variables on a dependent variable (with each 
path taking into account the effects of all the other variables). 

Due to the fact that data were collected on three levels - student-level, class-level and school 
level, multilevel modelling (Institute of Education, 2000) was applied. Multilevel modelling is 
a research tool applying multiple regression analysis, that investigates the interaction between 
variables on the individual (student) level and variables that describe the social groups to 
which the individuals belong (e.g, schools and classrooms). In this study, multilevel 
modelling was used to distinguish between the variance in mathematics achievement uniquely 
explained by student-level factors as opposed to the veuiance uniquely explained by the 
classroom and school-level factors and to investigate the individual effects of variables 
inserted in the model once the multilevel structure of the data is taken into account. As only 



one class per school was sampled, only two levels could be analysed and this was due to the 
original TIMSS-R design where class and school are considered one level. 

Results 

Results of the mathematics tests 

Overall, South African pupils achieved 275 points out of 800 (standard error, 6.8) in the 
mathematics test, whilst the international average was 487. This result is significantly below 
the mean scores of all other participating countries, including the two other African countries 
of Morocco and Tunisia as well as that of other developing or newly developed countries such 
as Malaysia, the Philippines, Indonesia and Chile (see Figure 1). The achievement in 
mathematics was dominated once again by the Asian countries of Singapore (at the top with 
604 scale points), Korea, Chinese Taipei, Hong Kong and Japan who achieved the top five 
scores. 




Figure 1: South African mean score compared to mean scores of selected TIMSS-R 
countries internationally 



The province with the highest average scale score for mathematics was Western Cape with 
381 scale points, but this was still significantly below the international mean score of 487. 
Northern Cape and Gauteng achieved the next highest scores with 318. Pupils from the 
Limpopo province scored well below all the other provinces with 226 points. 

The results of pupils' scores are given in Table 1 with reference to the regularity with which 
the language of the test is spoken at home as this is believed to have an influence on the 
overall results. The scores appear to reveal a trend that pupils that speak the language of the 
test more frequently also attain higher scores on the mathematics test. When comparing those 
pupils that almost always or always speak the language of the test to those that never speak 
the language of the test, the former achieve scores that are more than 140 points higher than 
the latter. 
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Table 1: Mean mathematics scores for South African pupils who always or almost 
always, sometimes or never speak the language of the test at home 







Always/almost 

always 




Sometimes 




Never 








N 


% of 

pupils Mean 


SE 


% of 

pupils Mean 


SE 


% of 
pupils 


Mean 


SE 


South African 
pupils' score 


8146 


23 370 


2.2 


53 259 


1.6 


24 


224 


1.8 



Source: Mullis et al, 2000. 



From a comparative analysis of South African pupils' results with other countries in TIMSS-R 
(see Howie, 2001), some interesting observations were made. More than 70% of pupils from 
South Africa, Indonesia, Morocco, Philippines and Singapore did not always speak the 
language of the test at home. Nonetheless, the mean achievement scores vary considerably 
across this group of countries and there are also some interesting trends in the data. Pupils in 
Malaysia generally did considerably better in mathematics than those from Indonesia. 
Nonetheless, there is a similar trend in both countries where pupils who never speak the 
language of the test at home, namely 9% in Indonesia and 10% in Malaysia, still appeared to 
outperform those who always or sometimes spoke the language of the test at home. It suggests 
therefore that the differences between language groups are not only dependent on language. 
Indonesia for instance is described as a highly diverse country with more than 600 languages 
and 200 million people (Baker & Prys-Jones, 1998, p. 375) and yet apparently their pupils do 
not appear to have been disadvantaged by writing the test in a second language. A similar 
pattern was also observed for Morocco and the Philippines in mathematics. In Singapore there 
does appear to be a difference, but yet those who never speak the language of the test at home, 
still outperform pupils from 33 other countries. 

Even looking at the other African countries, the scores of those never speaking the language 
at home are better in the case of Morocco for mathematics and are comparable for those from 
Tunisia. This issue needs to be explored further as it appears from the data, that the pupils 
from other developing countries do not seem to be disadvantaged by writing tests in their 
second or third language in mathematics or science, however it is not clear why this is. 
Important lessons for South Africa may lie in the answers. 

In South Africa, pupils who spoke either English or Afrikaans at home achieved higher scores 
(about 100 points above the national average) than those who did not and. What is 
illuminating is that children that spoke other languages at home (for example, Greek, 
Portuguese or Tamil) and therefore also learned in a second language, scored only 20 points 
on average less than first language speakers. However, children speaking African languages at 
home attained 100 points less than the other group of second language speakers. No group of 
pupils came close to attaining the international average for mathematics 

Results of the English language test 

In addition to the mathematics and science tests conducted in TIMSS-R, an English language 
proficiency test was included that aimed to assess pupils' writing related skills and language 
usage in English. The test comprised 40 items, which were multiple-choice items. Thirty of 
the forty items had four options, whilst the remaining 10 items had two answer options. 

The overall mean score for the language test was 17 out of 40 (42.5%; n= 8349). The 
minimum score attained was 0 and the maximum score 40. In general, the scores for boys and 
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girls were comparable. The scores varied across the nine provinces (see Figure 2) with the 
wealthiest and most urbanised provinces (Gauteng and Western Cape) attaining the highest 
overall scores. Thereafter, pupils in the Northern Cape achieved the highest of the remaining 
seven provinces. This is also consistent with the finding that Afrikaans speaking pupils 
performed reasonably well on the test as the majority of the Northern Cape pupils tested were 
Afrikaans speaking. Of some concern is the very low proficiency in English displayed by the 
Limpopo Province pupils, which was evident from the lowest scores attained of any province. 
In contrast to other provinces where the maximum score found was 34 points or higher, the 
maximum score recorded was 27 points out of 40 (67.5%). What is evident from the results is 
that the language performance follows a similar trend to that of mathematics, meaning that 
those provinces that performed better in mathematics also tended to perform better in the 
language test (see Figure 2). 




Figure 2: Average percentage correct achieved by South African pupils in mathematics 
and language per province 



As the test was designed for English second language speakers, it is not surprising that native 
English speakers performed the best of all language groups (25 points out of 40), although 
one might have expected the scores to have been higher given this fact. The Afrikaans 
speaking children attained the next highest score with 21 points out of 40. The scores were 
more or less consistent across the pupils whose main language was an African language. The 
Tswana and Southem-Sotho speaking pupils attained the highest scores out of those speaking 
African languages. This may reflect the urbanisation of the people speaking these two 
languages although other interpretations for this may also exist, for instance the emphasis on 
English language and communication skills in the former Bophutatswana, where most of the 
Tswana-speakers originate. 



PLS Exploration of the contextual factors on student, classroom and school 
Three hypothesised models on student, class and school level were analysed using Partial Least 
Square (PLS) analysis to explore the direct and indirect effects of individual variables on all 
three levels. The results of these analyses were scmtinised and thereafter the class and school 
level models were combined into one model and reanalysed. The main results are summarised 
here and the detailed explanation and discussion of these PLS results can be found in Howie 
(2002). As an illustration of a PLS model, the student level model is depicted in Figure 3 (see 
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appendix for explanation of the variables). The numbers on the arrows are the so-called beta- 
coefficients, indicating on a scale [-1,1] the strengths and the direction of the relationship. Sellin 
(1990) suggests that for large samples (like this sample of students) every coefficient with an 
absolute value >.07 represents a direct effect of the factor on mathematics achievement, whilst 
he recommends for models with smaller samples (like the school level model) a critical value 
of .10. 




Figure 3. An example of a PLS model: the student level model (see appendix for 
explanation of the variables). 

Student- level factors 

Data pertaining to the pupils' home background, their personal characteristics, their aptitude 
and competencies were explored. A high percentage of variance (50%) in the pupils' 
mathematics score was explained. Six factors were found to have a direct effect on South 
African pupils' performance in mathematics, namely the pupils' proficiency in English 
(engtest), their own self concept in terms of mathematics (selfcnp), the language pupils spoke 
at home (lang), their socio-economic status at home (SES), and whether or not they, their 
friends and their mothers thought that maths was important (mathim) and language of learning 
in the classroom {lanleam). 
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School-only level factors 

Some important aspects of school quality related to school leadership, parent involvement, 
school profile, physical resources, human resources, autonomy, learning environment and 
school administration were explored in the data from the school principal's questionnaire. Two 
important antecedents related to the type of community and the home language of the pupil, 
were included in the model. Sixty-two percent of the variance in the pupils' scores in 
mathematics could be explained by three factors at the school level, namely, the community 
where the school was located, the influence that the teachers union have on the curriculum, and 
an aggregated pupil variable, the extent to which the pupils in the class spoke the language of 
instmction as their first language. 

Classroom-level only factors 

From the mathematics teacher questionnaire, a number of classroom level factors were also 
explored and these resulted in including the following factors in the model: teachers' gender, 
teaching experience, teachers' level of education, time spent on activities, lesson preparation, 
teaching load, time on task, teachers' attitudes, success attribution, teachers' beliefs, teaching 
style, resources, limitations, and class size. In total, this model explained 46% of the variance 
in the pupils' mathematics scores by seven factors - the teachers attitudes, their beliefs about 
mathematics, the extent of their teaching and other workload, the size of the class they are 
teaching, their gender, resources and their dedication towards lesson preparation. 

Combined school-class level factors 

Because only one class per school was included in the study, class effects could not be studied 
independently from school effects. Therefore the school level model and the class-level model 
were combined and the predictors of mathematics achievement were selected from both 
models and combined with four aggregated student-level antecedent factors into one model. 
Therefore factors related to teachers' characteristics, pupils' home background, their aptitude, 
their attitudes, school quality, teaching requirements, curriculum quality and instmctional 
quality were all explored in one model. Finally, six factors were found that had direct effects 
on pupils' achievement in mathematics and that explained 27% of the variance in the 
mathematics score. These were the location of the school, class size, the attitude of the 
teacher, teachers' beliefs about mathematics, the teachers' workload (including teaching) and 
their dedication toward lesson preparation. 

Results from the Multilevel Analysis 

From the Partial Least Square analysis, the factors that had a direct effect on math 
achievement were identified and included into the multilevel analysis; language pupils spoke 
at home, socio-economic status, pupils' English test score, pupils' own self concept in terms of 
mathematics, whether or not they, their friends and their mothers thought that maths was 
important, language on the radio they most often listen to, size of the class the teachers were 
teaching, community where the school was situated, language spoken most often in the class 
by pupils and teachers, teachers' beliefs about mathematics, attitude of the teacher towards the 
profession, teachers' dedication toward lesson preparation, teachers' teaching load and 
teachers' total workload. A final variable was included because it was believed to be 
important from a political perspective, namely the number of pupils enrolled in a school. 
Ultimately 183 schools and 7 651 South African pupils were included in the multi-level 
analysis. 

In the first step in the analysis the only independent variable included in the analysis was the 
school of the learner (this is the so-called Null model). The Null-model in Table 2 shows that 
more than half of the variance in the mathematics achievement scores is situated on the school 
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level (55%) whilst 45% of the variance can be situated on student level. 

In the next step of the multilevel analysis the six student variables were entered successively 
in the model. The results are summarised in Table 2, column ‘Student model - Model 6’, in 
which the numbers represent the regression slopes associated with the various variables (e.g. 
5.35 is the regression coefficient associated with ‘home language’). Table 2 shows that all 
regression coefficients are significant, meaning that mathematics achievement tends to be 
better when the scores are higher on the variables home language, socio-economical status. 



Table 2: Multi Level analysis of the South African TIMSS data with the Math-test score 
as dependent variable (weighted data) 





Null Model 


Student Model 


Student-School 


Extended Student- 




(Model 0) 


(Model 6) 


(Model 15) 


school model 


Fixed effect 








(Model 16) 


student level 










Intercept 


288 


278 


299.5 


285.5 








3.27 


2.83 


Home language 






.88* 


0.79 


Socio-economic 
English test 




4.07 ** 


4.00** 


3 79** 


Self-concept 




-6.32 ** 


-6.29** 


-6.20** 


Importance of maths 




6.39 ** 


6.35** 


6.45** 


Radio language 




4.75 ** 


3.95** 


3.98** 


school level 










Status 






-17.27** 


-15.24** 


Beliefs about maths 






-4.46** 


-3.68** 


Location 






8.00** 


7.07* 


Class language 






2.59** 


2.55** 


Enrolment 






.00 


0.00 


Work time 






.52** 


0.49** 


Class size 






-.27 


-0.14 


Lesson planning 






8.02* 


7.16* 


Teaching time 






.10 


0.14 


Random ejfects 


6 520 (55%) 


2 451 


1 336 


1087 


School level variance 










Student level variance 


5 342 (45%) 


4 570 


4 560 


4535 




N/A 


1 340# 


132.80# 


1087# 


Difference in deviance 










Explained proportion of variance in math achievement 
(when compared with Null Model): 








School level variance 


.61 


.79 






Student level variance 


.44 


.50 







Notes-. Intercept represents the grand mean of the mathematics scores (differs slightly from the South African 
mean score of 275 on the international test due to the fact that a number of schools could not be included 
in the analysis) 

N = 7 651 pupils in 183 schools (one class per school); 

* t-value > 1,96, this resembles a confidence interval of 95%; 

** t-value > 2,58, this resembles a confidence interval of 99%; 

# The difference between the deviances is significant (p<.001). 

English tests, self-concept, importance of mathematics and radio language. These variables 
are described in the Appendix, however one explanatory remark should be made: as self- 
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concept has a negative scale, the negative regression coefficient should be interpreted as ‘the 
more difficulties a learner has with mathematics, the lower the achievement score . Howie 
(2002) illustrates these data with two examples to illustrate the influence of the language 
factor. The model predicts a difference of 57 points on the math achievement scale (with 
national mean score of 275 scale points) between a pupil that has the mean score on the 
English test (10 out of 40 points) and a pupil 24 points (the average achieved by native 
English speakers). As a second example, a pupil who sj)eaks an African language at home and 
never listens to an English radio station is predicted to score 25.6 points less on the 
mathematics achievement scale than a student who speaks English at home and always listens 
to an English radio station. It is therefore very clear that, if one only looks at variables at 
student level, there is a strong influence of language on achievement in mathematics. 

A final remark on the Student-model pertains to ‘difference in deviance (see Table 2). The 
deviance is a measure for the appropriateness as to whether a model is a good representation 
of the reality represented by the data. The fact that the difference in deviance between the 
Student model and the Null model is highly significant means that the Student-model is a 
highly significant improvement as compared to the Null-model. 



The next phase in the multilevel analysis was to enter the nine school level variables, resulting 
in the ‘Student-School model’, also called model 15 as the model is based on 15 independent 
variables. The results are summarised in the final column of Table 2. In total, 11 of the 15 
factors were found to be significant predictors of South African pupils' achievement in 
mathematics. Enrolment, class size, teaching time and home language appeared not 
significant in the Student-School model. Once the school-level variables were entered, the 
effect of home language, which was significant in the Student model, lost its significance as 
consecjuence of the school level variables such as location of the school. On the other hand, 
the English test score again appears to be one of the most significant (see Howie, 2002). 
Another result that can be derived from the full model is that SES is no longer significant (see 
Howie, 2002). The strength and significance of the school-level variables compensated for the 
pupil variables resulting in home language and SES losing their significance in the multilevel 
analysis. 

The quality of the Student-School model as the best one in representing the relationship 
between the independent variables and achievement can be shown from two other parts of 
Table 2. At first, the difference in deviance between the Student-School model (model 15) 
and the Student-model (model 6) is significant indicating that the Student-School model is 
superior. Furthermore, as can be concluded from the lowest part of Table 2, the strength of the 
Student-School model is also illustrated by the variance in mathematics achievement 
explained by this model, being 79% of the variance at school level, and 50% of the variance 

at student level. 

The extension of the Student-School-Model 

Due to the amount of the explained proportion variance on school level by the English test 
score {engtest), it was decided to extend the student-school-model with random slopes (i.e. 
with a random slope for each school) of the average English test score {engtest). The results in 
Table 2 indicate that this model is a significant improvement of the full model (Model 15), as 
the deviation from the full model is highly significant (p<.001; see Table 2). Noteworthy is 
that the data of Model 16 show that the extension of the full model with random slopes results 
in it that SES is no longer significant. This means that the other school variables explain the 
variance in SES (as was concluded for home language {hlang) when discussing the full 
model). Another observation in the data of Model 16 is that the inclusion of the random 
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slopes (i.e. per school its average value on engtest instead of taking the national average) 
results in lower estimates on all school variables, which shows that language proficiency is 
related to all school variables in another way. 



Each line is one school (weighted) 
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Average English Test score 



Note: the mean of the English test score is 17.01, but set zero in the figure. 

Figure 4 

Random slopes representing the predicted score on mathematics for each school/class (based 
on the English test scores) 

Figure 4 shows the final model graphically extended with a random slope for each class on 
the English test (engtest) with the other axis representing the mathematics achievement score 
(resulting in Model 16). 

If one looks at the pattern of the slopes in Figure 4, it would appear as if the impact of the 
English test (engtest) on mathematics achievement is less in classes with a low average score 
on the English test. In other words schools where pupils did poorly in the English test, their 
proficiency hardly made any difference to their mathematics score. Conversely, the better that 
classes of pupils performed on the English Test, the stronger the relationship of this outcome 
was with mathematics. In other words, the correlation between the English test and the 
mathematics score is higher for classes with an on average high score on the English test. 
There appears to be a curvilinear relationship between English and Mathematics, which means 
that language proficiency matters more when the English proficiency of classes is higher. 

Another observation should be made here. Figure 4 clearly shows that there are schools with a 
high average score on the English test and yet a low average performance on mathematics, 
combined with a low correlation between the two variables. This is an indication that there are 
in addition to English proficiency other variables (either within the model, e.g. location may 
be a candidate to investigate, or outside the present study) that are related to mathematics 
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achievement. One possibility is that some informal tracking of pupils may be implemented at 
schools. So, there may be classes where the majority of pupils speak English at home, but 
may be grouped into a low ability mathematics class. Possible evidence of this practice was 
highlighted during the data collection for TIMSS-R where in at least six schools principals 
prevented field workers from testing the sampled class on the grounds that it was a low 
achieving group and insisted that the "A" class be tested. Pupils from these schools were 
subsequently withdrawn from the TIMSS-R sample. Clearly further research on this is needed 
to ascertain whether this or other reasons are responsible for the trend observed in Figure 2. 

In conclusion, once all the predictors are added to the model, most of the school-level 
variance in pupils' achievement scores could be explained in the Student-School model. This 
is not the case for the student-level variance, as a large percentage of the variance on student 
level (50% of the 45% in the Null model) could not be explained by the predictors (including 
a number of language related variables) used in this model. This may be due to the fact that 
other variables that are not included in this study are important as well. For example, 
cognitive ability was not measured in this study, but was included in the Belgian-Hemish 
study as a national option. Van den Broek and Van Damme (2001) show that in Belgium this 
variable explains a great deal of variance on student level and explain in fact more than any 
other single variable in their multi-level model. Clearly more research is needed here for 
South Africa. 

However, the predictors did explain for the South African data a high percentage of the 
variance between schools. This means that a large part of the differences between schools in 
pupils' mathematics achievement can be attributed to these variables. The Student-School 
model indicates that significant predictors for how pupils in different schools perform in 
mathematics are the pupils' performance in the English test, the socio-economic status (to a 
lesser extent), the pupils' self concept, the pupils' perception of the importance of 
mathematics, their exposure to English, how pupils' maths teachers perceive their professional 
status, pupils' maths teachers beliefs about mathematics, the location of the school, the extent 
to which English is used in the classroom, the amount of time teachers spend working and the 
amount of time teachers spend in lesson planning. They are also significant predictors of how 
well pupils' perform in the same school (within-school variance), but to a lesser extent. 
Noteworthy is that two of these variables have a negative effect, teachers' perception of their 
status and their beliefs about mathematics. The stronger the teachers' ideas about mathematics 
and the perception about the status of the profession are, the poorer their pupils perform in 
mathematics. This observation should not be looked at in isolation, but in conjunction with 
the other variables that have a significant effect on mathematics achievement, but further 
discussion is beyond the scope of this article. 

Conclusion and Reflections 

The first research question addressed the effect of language on South African pupils’ 
performance in mathematics. This research shows that in South Africa, pupils tended to 
achieve higher scores in mathematics when their language proficiency in English was higher 
and were more likely to attain low scores in mathematics when their scores on the English test 
were low. Those that spoke English or Afrikaans at home tended to achieve higher scores in 
mathematics. Alternatively, children from homes where African languages were used were 
more likely to achieve lower scores. Pupils in classes where the pupils and the teachers mostly 
interacted in the official media of instruction (English or Afrikaans) were more likely to 
achieve better results in mathematics. 
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A key finding from this research is that pupils who spoke either English or Afrikaans at home 
achieved higher scores in both the mathematics and the English tests than those who did not. 
There is often a mismatch in many cases regarding the language of the pupil and the language 
of the teacher. With the increasing heterogeneity of South Africa's schools in culture and 
language, offering children the opportunity to learn in their own home language is becoming 
increasingly challenging, as is also illustrated by other (qualitative) studies in South Africa 
about the relationship between mathematics and language (see e.g. Adler, 1998; Setati & 
Adler, 2000; Setati, Adler, Reid & Bapoo, 2001). 

With regard to language another key finding was that the average English test score was very 
low and the majority of pupils' English language proficiency was poor. Pupils speaking 
African languages had very low English language proficiency as compared to other second 
language learners. Furthermore, there is no room for complacency as native English speakers 
as a group did not perform substantially better than the Afrikaans-speaking group given that 
this was an English second language test. The low proficiency may be linked to another 
finding from the research (see Howie, 2002) that more than two-thirds of pupils had very few 
books in the home It is clear that an urgent and intensive intervention is needed in both 
English as a discipline in the schools and as a medium of instruction. Above all, children need 
to be encouraged to read in their leisure time, but they also need to be given opportunities to 
write in English more so than presently so that they become familiar in articulating thoughts 
and knowledge in English and this should be targeted to all children, not only the second 
language speakers. 

The second research question focused on how does the effect of language relate to other 
background variables collected at school and student level. 

The Student-model (model 6) shows that in addition to language of the home and English 
proficiency other variables such as SES, self-concept of the learner (about having difficulty 
with mathematics) and importance of mathematics (according to mother, friends and the 
learner) are also related to mathematics achievement. 

The Student-School model shows the influence of the location of the school in rural or urban 
areas on mathematics achievement, which is not surprising given the under-development in 
rural areas in South Africa. However, as 50% of South Africa's population live in rural areas, 
the fact that students attending school in rural areas perform worse in mathematics than those 
attending schools in urban areas should be of serious concern to the education and other 
authorities and policy-makers. An interesting outcome was the strength of teachers' attitudes, 
beliefs and dedication as predictors of pupils' achievement. Teachers with strong mathematical 
pedagogical beliefs and who believed that mathematics is primarily a formal way of 
representing the real world; that mathematics is primarily a practical and structured guide for 
addressing real situations; if students are having difficulty an effective approach is to given 
them more practice by themselves during the class; that more than one representation (picture, 
concrete material, symbol set etc) should be used in teaching a maths topic; the mathematics 
should be learned as sets of algorithms or rules that cover all possibilities; and that basic 
computational skills on the part of the teacher are sufficient for teaching secondary school 
maths; were also those whose pupils were more likely to achieve lower results. This may mean 
that the teachers with stronger beliefs have less content knowledge and less understanding about 
the philosophy of mathematics perhaps due to their training or lack thereof. This may also result 
in teaching pupils through rote learning whilst reporting in a way that reflects the philosophy of 
the new South African curriculum, namely pupil-centred and problem-based teaching. On the 
other hand, teachers that had weaker beliefs may reflect a group of teachers who had good 
training, have a good content knowledge and have confidence in their preparation to teach, but 
stick to traditional teaching methods in contrast to those advocated by the new curriculum. 
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Pupils of such teachers may have performed better on the TIMSS-R test, a test, which has been 
criticised by proponents of realistic mathematics in the Netherlands as too traditional in its 
design and content. Further investigation of teachers' beliefs in relation to pupils' achievement is 
needed. 

Teachers with feelings of being appreciated by society and their pupils were more likely to 
produce pupils with lower results. This could be due to the fact that these are teachers in more 
rural areas who are highly regarded by their communities, as they are well educated compared 
to others in those communities. However, schools in these same communities are also poor 
and lack resources and pupils come from poor homes and have much exposure to the 
languages of learning than their urban counterparts. So, although teachers feel affirmed, they 
are also challenged by the conditions in their schools, which results in pupils attaining lower 
scores. On the other hand teachers who feel less affirmed work in better conditions in urban 
areas and where their pupils come from different background. Teachers' commitment appears 
to play a key role in pupils' performance. Pupils whose teachers spent more time with work 
related activities at school tended to achieve higher results in mathematics, although the 
amount of time that, teachers spent actually teaching had no effect. Likewise, the amount of 
time that teachers spent preparing lessons resulted in pupils being more likely to achieve 
higher results. 

The lack of effect of class size as a predictor of achievement confirms previous findings in 
studies of developing countries. In the case of South Africa, those classes with large numbers 
of pupils (on average 50 pupils) are also those at schools with poor conditions described 
earlier and therefore it is possible that a type of bottom effect is felt here and therefore, the 
effect of the actual number of pupils is slight. 

Finally, when all other factors were considered, the most significant factor was the English 
test score and this was highlighted in the final extended model where the strength of the effect 
could be clearly seen. The data of Model 16 (the full model with random slopes for the 
English test) showed that the extension of the full model with random slopes results in SES no 
longer being significant. The strength and significance of the school-level variables 
compensated for the student variables resulting in home language and SES losing their 
significance in the multilevel analysis 

These findings are significant against the background of the situation in many South African 
schools, particularly in those where there are African pupils taught by African teachers. In 
these schools the conditions are the worst: limited resources and facilities, large percentages 
of under-qualified teachers, pupils from poor socio-economic backgrounds and instruction 
occurs in a secondary language. 

The findings give rise to a number of reflections on the relationship between language and 
achievement in relation to the language policy implemented by the South African 
government. The difficulty of not being able to communicate fluently in a common language 
is leading to increased frustration for the teacher, disorientation on the part of the child, a slow 
rate of learning, disciplinary problems and teacher centred instruction. Although teachers are 
aware of the national language policy they may have very different interpretations of it 
(Setati, 1999). The majority of parents, pupils and teachers perceive English as the gateway to 
opportunities more globally and therefore many want the pupils to participate in their 
education through the medium of English. The implementation of the language policy needs 
reassessing regarding its feasibility and the desires of the community. Unless, some hard 
decisions are taken, and quickly, by decision-makers, pupils and teachers will continue to 
struggle and pupils will under-perform in mathematics and other subjects. If, as the majority 
of parents seem to desire, English is increasingly used then the necessary support mechanisms 
need to be put into place including intensive language training of second language teachers 
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who will be teaching through the medium of English. On the other hand, if however the 
decision is made to teach in mother tongue beyond grade 5, the ramifications are enormous 
considering that in urban environments, almost all the official languages are found as well as 
there are increasingly more foreign languages (from elsewhere in Africa and beyond). 
Segregating children and indeed teachers in terms of language (as clearly offering 11 
languages of learning will be impossible within a single school) will be an enormous task and 
may reverse the cultural integration beginning to take place. Of course, in rural areas where 
one language is almost always clearly dominant, it may be more feasible. The danger of 
considering only one approach in the language policy, albeit an important one, (for example 
only considering cultural identity or the political perspective); is that the multiple functions of 
schooling will be ignored. For, the curriculum and pupils' education have to fulfil the desires 
of society namely, that of educating them towards being a responsive citizen in a democratic 
society, of attaining certain basic knowledge and skills, being prepared for the workplace 
and/or further education and acquiring adequate social and interpersonal skills. The 
government has to take a firm lead in finding the appropriate balance between these 
perspectives, as if this is left to the schools, the status quo will remain. 

In conclusion, the strength of the language component represented in a number of variables 
that have strong effects on mathematics achievement is clear. Moreover, the dedication of the 
teacher matters with regard to their pupils' achievement and the location of the school is 
further an important predictor of South African pupils' achievement in mathematics. In the 
past few years, some progress has been made to address shortcomings in South African 
schools. Although significant progress has been made with regard to administrative 
restructuring, policy development and infra-structural improvements nonetheless, the quality 
of education that the majority of pupils are receiving is far from satisfactory. The study 
(Howie, 2002) highlighted the most significant predictors of mathematics (of which language 
is only one) within the scope of the data available and has raised a number of questions and 
issues that are believed to be pertinent to the future development of South African education, 
in particular in mathematics. The challenges abound within the education system of this 
country and in addition to the issues of access and equity the most important challenge 
awaiting now is that of quality. 
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Appendix 1 



LATENT 

VARIABLES 


MANIFEST 

VARIABLES 


DESCRIPTION 


SCORING 


MATHSCR 


MATH 


Student mean score on 
TIMSS-R mathematics test 


Score out of 800 points 


ENGTEST 


TOTSCORE 


Student mean score on 
English language proficiency 
test 


Score out of 40 points 


RACE 


RACE 1 


Race of student: African, 


L African 






Coloured, Indian, White, 
Asian 


2. Asian 

3. Coloured 

4. Indian 

5. White 


AGE 


AGEl 


Age of student 


Number of years 


LANG 


RALANG 1 


Language on favourite radio 


1. All other langs 






station 


2. Afrikaans 

3. English 


HLANG 


HOMELANG 


Language spoken most often 
at home 


0. Other languages 

1 . African languages 

2. Afrikaans 

3. English 


HOME 


FAMSIZE 


Number of people living in 
the home 


- 




PARENT 


Whether students have 2 


o 

!1 

o 




(COMPOSITE) 


parents 


1 = yes 


BOOK 


BOOKS 


Number of books in the 
home 


1. 0-10 

2. 11-25 

3. 26-100 

4. 101-200 

5. more than 200 


SES 


POSSES 10 


Computer, electricity, tap 


0. no 




(Composite) 


water, tv, CD player, radio, 
own bedroom, flush toilets, 
car (9 items) 


1. yes 


CLASSENV 


QUIETCL 


Class climate items 


Scale of (-)l-4 (+) 




OBEDCL 


Extent to which the students 
report that students in their 
classes are quiet in class, are 
obedient in class 


Strongly agree to strongly disagree 
Scale of(-)l-4 (+) 

Strongly agree to strongly disagree 
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LATENT 

VARIABLE 

S 


MANIFEST 

VARIABLES 


TIMSS-R 

VARIABLE 

S 


description 


SCORING 


LANLEARN 


LANLEARN 

(Composite) 




Extent to which both 
student and teacher 
speak language of 
instruction in maths 
class 


1. language Spoken 
at home if not 
Engli sh/ Afrikaan 
s 

2. sometimes 

Eng li sh/ Afrikaan 
s 

3. most of the time 
English/ Afrikaan 
s 

4. always English/ 
Afrikaans 


MATHIMPT 


MATHIMPT 

(Composite) 


BSBGMIP2 

BSBGSIP3 

BSBGFIP2 


Extent to which 
students', Mother, 
friends think that maths 
is important (3 items) 


Scale of(+)l-4(-) 
strongly agree to 
strongly disagree 


ASPIRE 


SELFED_1 


GENEDSE 


Aspirations to education 


1. some secondary 

2. finished 
secondary 

3. finished 
technikon 

4. some university 

5. finished 
university 


ATTEND 


ATTEND 

(Composite: 

SELFSKIP, 

FRSKIP) 


BSBGSSKP 

BSBGFSKP 


Extent to which student 
or students' friends skip 
school (2 items) 


Scale of (+) 0-4 (-) 
never, once or twice, 
three or four times, 
five or more 


SELFCNPT 


DIFMATH 


MATHMYT_ 

1-5 


The extent to which 
student reports having 
difficulty with maths (5 
items) 


Scale of(-)l-4(+) 
Strongly agree to 
strongly disagree 


SUCATTRB 


LUCK 


MATHDOW 

2 


If the student attributes 
success to Luck 


Scale of (-)1“4 (+) 
Strongly agree to 
strongly disagree 


ATTITUDE 


BORE 


MATHBORE 


If student finds maths 
boring 


Scale of (-)l-4 (+) 
Strongly agree to 
strongly disagree 
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